System and method for capturing augmented reality electronic signatures

ABSTRACT

A system and method for more accurately and easily recording signatures electronically/digitally. The preferred signing experience is generating a “wet” signature by putting a pen to paper as a pen and paper are ubiquitous and portable as well as familiar to most people. The present invention preserves this experience while combining it with a digital rendering of a document using conventional mobile communication devices having a camera feature in order to record a wet signature of a user and store a digital representation of the document that includes the recorded wet signature in the signature block of the document.

FIELD OF THE INVENTION

The application related to the field of electronic signatures and inparticular to the capturing of signatures with a mobile personal deviceassistant device.

BACKGROUND OF THE INVENTION

As more paper-based workflows are managed electronically, such apaperless offices, the need for electronic signatures is increasing.Conventional solutions attempt to replicate the experience of signing adocument through the emulation of conventional pen to paper, i.e.,“wet,” signatures using, for example, touch screen, computer pen,computer mouse or other specialized input device. A system requiring aspecialized input device limits the proliferation and adoption of suchsystems. Using a finger or other device on a touch screens requires thespecialized touch screen device and is also slow, cumbersome, unfamiliarto a typical user and frequently results in a signature that is a poorrepresentation of a wet signature. Similarly, the use of specializedhardware such as computer pens are unfamiliar to the typical user as thesignature often is misaligned with where the pen touches the screen, andthe touch screen sensitivities result in “shaky” signatures. Inaddition, as described above, the requirement of a specialized pen is ahindrance to adoption of such systems. The use of a computer mouse isalso a poor substitute of a wet signature as the use of a mouse to signa document is unfamiliar and results in signatures that often bearlittle resemblance to wet signatures. Such poor representations ofsignatures results in authentication concerns.

Alternatively, conventional electronic signatures can be typewritten,such as the name between backslash characters, “/signature/”. However, aproblem with this technique is that such signatures are easy to forgeand therefore present authentication concerns.

SUMMARY OF THE INVENTION

An embodiment of the invention is a method for associating a digitalrepresentation of a wet signature with a digital document that is adigital representation of a document, comprising the steps of:initiating a signature application on a mobile communication devicehaving a camera and a display; displaying the digital document on saiddisplay of said mobile communication device; positioning said camera ofsaid mobile communication device over the wet signature wherein arepresentation of the wet signature is positioned in a signature blockof the digital document by said signature application; capturing the wetsignature by said camera; and storing, by said signature application, anexecuted version of the digital document with the wet signature digitalrepresentation in said signature block of the digital document.

The features and advantages described in the specification are not allinclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes, and may not have been selectedto delineate or circumscribe the inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of components of the invention in accordancewith an embodiment of the present invention.

FIG. 2 is a detailed illustration of a mobile communication device inaccordance with an embodiment of the present invention.

FIG. 3 is a flowchart of a method in accordance with an embodiment ofthe present invention.

FIG. 4 is an illustration of components of the invention in accordancewith an embodiment of the present invention.

FIG. 5 is an illustration of components of the invention in accordancewith an embodiment of the present invention.

FIG. 6 is an illustration of components of the invention in accordancewith an embodiment of the present invention.

FIG. 7 is an illustration of components of the invention in accordancewith an embodiment of the present invention.

The figures depict various embodiments of the present invention forpurposes of illustration only. One skilled in the art will readilyrecognize from the following discussion that alternative embodiments ofthe structures and methods illustrated herein may be employed withoutdeparting from the principles of the invention described herein.

DETAILED DESCRIPTION OF THE INVENTION

A preferred embodiment of the present invention is now described withreference to the figures where like reference numbers indicate identicalor functionally similar elements. Also in the figures, the left mostdigits of each reference number corresponds to the figure in which thereference number is first used.

FIG. 1 is an illustration of components of the invention in accordancewith an embodiment of the present invention. In one embodiment theinvention includes a mobile communication device (MCD) 102, paper orother material that can be written upon, for ease of discussion thiswill be referred to herein as paper 104, but can include any material orobject that can be written upon, e.g., desk, hand, blackboard,whiteboard, etc. FIG. 1 also includes a writing device 106, e.g., a pen,marker, pencil, etc.

FIG. 2 is a detailed illustration of a mobile communication device 102in accordance with an embodiment of the present invention. Examples of awireless mobile communication device (MCD) 102 include a cellular phone,personal device assistant (PDA), smart phone, pocket personal computer(PC), laptop computer, smart watch or other devices having a processor,communications capability and are easily transportable, for example.Examples of applications include applications available for the iPhone™that is commercially available from Apple Computer, Cupertino, Calif.,applications for phones running the Android™ operating system that iscommercially available from Google, Inc., Mountain View, Calif.,applications for BlackBerry devices, available from RIM, Ontario Canada,or applications available for Windows Mobile devices, available fromMicrosoft Corp., Redmond, Wash. Such devices are examples of the MCD102.

In an embodiment the MCD 102 includes a communications unit 206, aprocessor 208, an input/output device 212, a camera 216 and amemory/storage device 204. The MCD 102 has an operating system and caninclude various applications either integrated into the operating systemor stored in the memory/storage device 204 and executed by the processor208. In an embodiment, the memory device 204 includes a data storagemodule 222, a software module (signature app 220), among other softwareand programs, for example, that is executed on the MCD 102. The softwaremodule can be an application, such as an iPhone™ or Android™-basedapplication. For ease of discussion this software module will bereferred to herein as a signature application 220. The signatureapplication 220 can be stored on the MCD 102 and can be part of theembedded software of the MCD 102 (e.g., iPhone) or integrated into theMCD's 102 operating system. Alternatively, the application can bedownloaded via a wired or wireless system, e.g., using Apple's iTunes™.In addition, in an embodiment the software module can be part of any ofa variety of software paradigms, e.g., software as a service (SaaS),cloud computing. For ease of discussion, the following description willbe based on the model of the software module being an application thatis stored in the MCD memory module 104 and executed by the MCD processor208.

FIG. 3 is a flowchart of a method in accordance with an embodiment ofthe present invention. Some embodiments of the operation of theinvention will be described with reference to FIGS. 4-7.

FIG. 4 is an illustration of components of the invention in accordancewith an embodiment of the present invention. As described above, thepresent invention is a system and method for more accurately and easilyrecording signatures electronically. The preferred signing experience isgenerating a “wet” signature by putting a pen to paper, as a pen andpaper are ubiquitous and portable as well as familiar to most people.The present invention preserves this experience while combining it witha digital rendering of a document (referred to herein as a digitaldocument, the digital document need not be created from a physicaldocument, that is, it can be a document created using conventional wordprocessors, e.g., Microsoft Word, and directly stored into a computerreadable memory). The digital document is to be signed in a manner thatovercomes the drawbacks of conventional electronic signing system andmethods, as described above. In particular, MCDs (102) have becomecommon and many MCDs (102) include a camera 216. The present inventionuses an MCD 102 to record the signature and/or act of signing andsuperimposes the signature into the document. The signer or agent canrecord and save the superimposed signature with the document along withauthentication information.

With reference to FIG. 3, a user can initiate 302 the signatureapplication 220 on the MCD 102. The steps set forth in FIG. 3 areillustrative and the steps do not necessarily need to occur in the ordershown. The signature application 220 can operate along with a documentreading and displaying software, e.g. Microsoft Word, Adobe, etc, ordocument reading and displaying software can be part of the signatureapplication 220. The document reading/displaying feature provides adigital rendering of the document on the screen of the MCD 102. Thesignature application 220 can include security protections to enableaccess only when the security protections are satisfied. Examples ofsuch security include the use of a password, voice recognition, retinalscan, particular location, e.g., within a one mile radius of Fenwick &West LLP in Mountain View, Calif., fingerprint recognition possiblyreadable only by a device with a specific identifier, e.g., a serialnumber, SIM (subscriber identity module) card, phone number, etc. Thesignature application 220 utilizes the MCD camera 216 and identifies thesignature block area of the document. In an embodiment, the signatureapplication 302 receives an output from the camera 216, for example arepresentation of the signature that will be received by the camera 216when the camera is activated, such as a representation similar to thatwhich may be displayed in a viewfinder, and superimposes the cameraoutput onto the signature block area of the document. In alternateembodiments, other portions of the document can be filled in by theuser, e.g., the date or other information at different areas of thedocument. The signature application can perform these functions using avariety of conventional techniques, e.g., using text field entry,drawing on a touchscreen in various colors, loading previously recordeddata, e.g., addresses, system clock for time/date.

With reference to FIG. 4, the user signs 304 the paper 104 using thewriting device 106. As seen in FIG. 4, the paper (or other writingsurface) signed need not include the text of the document, e.g., thepaper can be blank prior to signing. In this embodiment the signaturecan occur prior to the initiation 302 of the signing application 220.

FIG. 5 is an illustration of components of the invention in accordancewith an embodiment of the present invention. The user places the MCD 102such that the camera 218 captures the signature on the paper 104 andthat the signature appears in the proper location 502 in the electronicdocument displayed on the MCD 102. The signature application 220captures 306 the signature, preferably when the signature is displayedin the proper location 502 in the electronic document. In an embodimentthe signature capture can occur by the pressing of a button or touchinga particular area, e.g., touching the screen on the MCD 102, or using averbal command, for example. The document is stored 308 in the MCDmemory module 204 with the signature and additional data such as thetime/date of the signature capture, the location of the signaturecapture (which can be based upon location identification features of theMCD 102, for example), the authentication information provided by thesignature application security features (for example, a password,fingerprint, voice identification or other information that is providedin response to a security information request by the signatureapplication), MCD 102 identification information, retina scan, locationinformation (using global positioning satellite (GPS) information orother location determination information/systems such as cellular towertriangulation based on signal strengths from multiple cellular towers),device identifier, phone number, etc. This information can be storedwith the document in the memory module 204, for example as metadataand/or can be stored as a separate file.

The additional data can assist in authenticating the signature. Asdescribed above, access to the signature application 220 can requiresatisfying the security features of the signature application 220 whichcan assist in authenticating the user of the signature application, theuse of a particular MCD 102 as identified by the data can provideadditional information to assist in authenticating the signature. Theremay also be security features required to access the MCD 102 whichprovides additional security and authentication information.

In addition, since the user uses a writing device 106 and writingsurface 104 to create a wet signature, which is consistent with thesignature process that the user is accustomed to, the signature itselfprovides a measure of authentication as the signature can be comparedwith other signatures of the user. In addition, the additional datacollected, as described above, such as the time/date of the signaturecapture, the location of the signature capture (which can be based uponlocation identification features of the MCD 102, for example), theauthentication information provided by the signature applicationsecurity features, and/or MCD 102 identification information, forexample, also assist in authenticating the signature and the user(person signing the document).

FIG. 6 is an illustration of components of the invention in accordancewith an embodiment of the present invention. In another embodiment, thesignature application 220 captures a video of the signature creation.This can be done using a video capture function of the MCD 102. In oneembodiment, after initiation 302 of the signature application 220, theuser positions the MCD 102 such that the signature on the paper 104 ispositioned such that the signature appears at the proper location 602 ofthe document representation. The user can select a video capture modeand initiate 306 the video capture. Using the writing device 106, theuser signs 304 the document which is captured as a video signatureand/or a static digital signature and the captured video and/or staticsignatures are stored 308 in the memory device 104 as part of thedocument file and/or as a separate file, for example. In an alternateembodiment, the document and additional information can be transmittedto a storage device (not shown) that is remote from the MCD 102 using,for example, conventional communication methods, e.g., wirelesstelephone communication network, Bluetooth communication protocol, aWiFi network, etc. FIG. 7 is an illustration of components of theinvention in accordance with an embodiment of the present invention. InFIG. 7, the signature is shown in the signature block of the document.

In addition to the information described above that can be stored aspart of the signature capture event, the video provides additionalinformation that can be used to authenticate the signature and the user.

Reference in the specification to “one embodiment” or to “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiments is included in at least oneembodiment of the invention. The appearances of the phrase “in oneembodiment” or “an embodiment” in various places in the specificationare not necessarily all referring to the same embodiment.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps (instructions)leading to a desired result. The steps are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical, magnetic or opticalsignals capable of being stored, transferred, combined, compared andotherwise manipulated. It is convenient at times, principally forreasons of common usage, to refer to these signals as bits, values,elements, symbols, characters, terms, numbers, or the like. Furthermore,it is also convenient at times, to refer to certain arrangements ofsteps requiring physical manipulations or transformation of physicalquantities or representations of physical quantities as modules or codedevices, without loss of generality.

However, all of these and similar terms are to be associated with theappropriate physical quantities and are merely convenient labels appliedto these quantities. Unless specifically stated otherwise as apparentfrom the following discussion, it is appreciated that throughout thedescription, discussions utilizing terms such as “processing” or“computing” or “calculating” or “determining” or “displaying” or“determining” or the like, refer to the action and processes of acomputer system, or similar electronic computing device (such as aspecific computing machine), that manipulates and transforms datarepresented as physical (electronic) quantities within the computersystem memories or registers or other such information storage,transmission or display devices.

Certain aspects of the present invention include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present inventioncould be embodied in software, firmware or hardware, and when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by a variety of operating systems. Theinvention can also be in a computer program product which can beexecuted on a computing system.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for thepurposes, e.g., a specific computer, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer readable storage medium, such as, but is notlimited to, any type of disk including floppy disks, optical disks,CD-ROMs, magnetic-optical disks, read-only memories (ROMs), randomaccess memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards,application specific integrated circuits (ASICs), or any type of mediasuitable for storing electronic instructions, and each coupled to acomputer system bus. Memory can include any of the above and/or otherdevices that can store information/data/programs and can be transient ornon-transient medium. Furthermore, the computers referred to in thespecification may include a single processor or may be architecturesemploying multiple processor designs for increased computing capability.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may also be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the method steps. The structure for a variety ofthese systems will appear from the description herein. In addition, thepresent invention is not described with reference to any particularprogramming language. It will be appreciated that a variety ofprogramming languages may be used to implement the teachings of thepresent invention as described herein, and any references herein tospecific languages are provided for disclosure of enablement and bestmode of the present invention.

In addition, the language used in the specification has been principallyselected for readability and instructional purposes, and may not havebeen selected to delineate or circumscribe the inventive subject matter.Accordingly, the disclosure of the present invention is intended to beillustrative, but not limiting, of the scope of the invention, which isset forth in the claims.

While particular embodiments and applications of the present inventionhave been illustrated and described herein, it is to be understood thatthe invention is not limited to the precise construction and componentsdisclosed herein and that various modifications, changes, and variationsmay be made in the arrangement, operation, and details of the methodsand apparatuses of the present invention without departing from thespirit and scope of the invention as it is defined in the appendedclaims.

1. A method for associating a digital representation of a wet signaturewith a digital document, comprising the steps of: initiating a signatureapplication on a mobile communication device having a camera and adisplay; displaying the digital document on said display of said mobilecommunication device; positioning said camera of said mobilecommunication device over the wet signature wherein a representation ofthe wet signature is positioned in a signature block of the digitaldocument by said signature application; capturing the wet signature bysaid camera; and storing, by said signature application, an executedversion of the digital document with the wet signature digitalrepresentation in said signature block of the digital document.
 2. Themethod of claim 1, further comprising the step of: identifyingauthentication information at substantially the same time as saidcapturing of the wet signature.
 3. The method of claim 2, wherein saidauthentication information includes at least one of a location of themobile communication device, a mobile communication device identifier, atime of the capturing of the wet signature, or information provided inresponse to security requests of said signature application.
 4. Themethod of claim 1, further comprising the step of: capturing a signaturevideo by the camera of the creation of the wet signature; wherein saidstoring step includes storing said signature video.
 5. The method ofclaim 4, wherein said executed version of the electronic document isstored as a first file and said signature video is stored as a secondfile, wherein said first file is associated with said second file. 6.The method of claim 4, wherein said signature video is part of saidexecuted version of said digital document.
 7. The method of claim 4,further comprising the step of: identifying authentication informationat substantially the same time as said capturing of the wet signature.8. The method of claim 7, wherein said authentication informationincludes at least one of a location of the mobile communication device,a mobile communication device identifier, a time of the capturing of thewet signature, or information provided in response to security requestsof said signature application.
 9. A system for associating a digitalrepresentation of a wet signature with a digital document, comprising: amobile communication device having a camera, a display, a storagedevice, a processor and a signature application that is executed by saidprocessor; said display displays the digital document; said camera ispositioned over the wet signature wherein a representation of the wetsignature is positioned in a signature block of the digital document bysaid signature application; said camera captures the wet signature; andsaid mobile communication device storing an executed version of thedigital document with the wet signature digital representation in saidsignature block of the digital document.
 10. The system of claim 9,further comprising: said mobile communication device identifyingauthentication information at substantially the same time as saidcapturing of the wet signature.
 11. The system of claim 10, wherein saidauthentication information includes at least one of a location of themobile communication device, a mobile communication device identifier, atime of the capturing of the wet signature, or information provided inresponse to security requests of said signature application.
 12. Thesystem of claim 9, further comprising: said camera capturing a signaturevideo of the creation of the wet signature; and said mobilecommunication device application storing said signature video.
 13. Thesystem of claim 12, wherein said executed version of the electronicdocument is stored as a first file and said signature video is stored asa second file, wherein said first file is associated with said secondfile.
 14. The system of claim 12, wherein said signature video is partof said executed version of said digital document.
 15. The system ofclaim 12, further comprising: said mobile communication identifyingauthentication information at substantially the same time as saidcapturing of the wet signature by the camera.
 16. The system of claim15, wherein said authentication information includes at least one of alocation of the mobile communication device, a mobile communicationdevice identifier, a time of the capturing of the wet signature, orinformation provided in response to security requests of said signatureapplication.
 17. A non-transitory computer readable medium comprisingcomputer executable code which when executed by a computer performs thesteps of associating a digital representation of a wet signature with adigital document, comprising the steps of: initiating a signatureapplication on a mobile communication device having a camera and adisplay; displaying the digital document on said display of said mobilecommunication device; positioning said camera of said mobilecommunication device over the wet signature wherein a representation ofthe wet signature is positioned in a signature block of the digitaldocument by said signature application; capturing the wet signature bysaid camera; and storing, by said signature application, an executedversion of the digital document with the wet signature digitalrepresentation in said signature block of the digital document.
 18. Thenon-transitory computer readable medium comprising computer executablecode of claim 17, further comprising the step of: identifyingauthentication information at substantially the same time as saidcapturing of the wet signature.
 19. The non-transitory computer readablemedium comprising computer executable code of claim 18, wherein saidauthentication information includes at least one of a location of themobile communication device, a mobile communication device identifier, atime of the capturing of the wet signature, or information provided inresponse to security requests of said signature application.
 20. Thenon-transitory computer readable medium comprising computer executablecode of claim 17, further comprising the step of: capturing a signaturevideo by the camera of the creation of the wet signature; wherein saidstoring step includes storing said signature video.